# Incremental Pre-training

A Vietnamese-optimized model based on Llama2-chat 7B, significantly improving Vietnamese language processing through incremental pre-training and an efficient tokenizer

ZhiLu is a financial large language model developed based on Chinese Alpaca2-13B, achieving capability leaps through massive incremental pre-training with Chinese and English corpora and high-quality instruction data alignment, with a focus on enhancing performance in the financial domain.

CodeS-7B is a large language model optimized for SQL generation, based on incremental pre-training of StarCoderBase-7B, supporting a maximum context length of 8,192 tokens.

A large-scale pre-trained model with 13 billion parameters based on the LLaMa architecture, optimized for Chinese tokenization, completing 110 billion tokens of incremental pre-training in Chinese and English, significantly improving Chinese generation and comprehension capabilities

VMware's BERT base model optimized for technical domains, enhanced with incremental pre-training to improve the handling of proprietary terminology